Skip to content

[PLUGIN-1835] Add OracleErrorDetailsProvider#532

Merged
psainics merged 1 commit intodata-integrations:developfrom
cloudsufi:fem/oracledb
Jan 29, 2025
Merged

[PLUGIN-1835] Add OracleErrorDetailsProvider#532
psainics merged 1 commit intodata-integrations:developfrom
cloudsufi:fem/oracledb

Conversation

@psainics
Copy link
Contributor

@psainics psainics commented Dec 23, 2024

ErrorDetailsProvider - Oracle DB [Source|Sink] plugin

Jira : PLUGIN-1835

Description

Implement Program Failure Exception Handling in Oracle DB Source/Sink plugin to catch known errors

Code change

  • Added OracleErrorDetailsProvider.java
  • Modified DBUtils.java
  • Modified OracleSink.java
  • Modified OracleSource.java

E2E Changes

  • Adding Open and capture logs step when pipeline is expected to fail.

Tests

  • Test Case (Violate check constrain)
Raw Logs

2025-01-29 04:09:03,101 - ERROR [Executor task launch worker for task 0.0 in stage 0.0 (TID 0):o.a.s.u.Utils@98] - Aborting task
io.cdap.cdap.api.exception.WrappedStageException: Stage 'Oracle2' encountered : io.cdap.cdap.api.exception.ProgramFailureException: Error occurred in the phase: 'Writing' with sqlState: '23000', errorCode: '1', errorMessage: ORA-00001: unique constraint (C##BUILDUSER.PK_AGE) violated

https://docs.oracle.com/error-help/db/ora-00001/
  at io.cdap.cdap.etl.common.ErrorDetails.handleException(ErrorDetails.java:77)
  at io.cdap.cdap.etl.spark.io.StageTrackingRecordWriter.close(StageTrackingRecordWriter.java:67)
  at org.apache.spark.internal.io.HadoopMapReduceWriteConfigUtil.closeWriter(SparkHadoopWriter.scala:373)
  at org.apache.spark.internal.io.SparkHadoopWriter$.$anonfun$executeTask$1(SparkHadoopWriter.scala:145)
  at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1539)
  at org.apache.spark.internal.io.SparkHadoopWriter$.executeTask(SparkHadoopWriter.scala:135)
  at org.apache.spark.internal.io.SparkHadoopWriter$.$anonfun$write$1(SparkHadoopWriter.scala:88)
  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
  at org.apache.spark.scheduler.Task.run(Task.scala:136)
  at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548)
  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1505)
  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.lang.Thread.run(Thread.java:829)
Caused by: io.cdap.cdap.api.exception.ProgramFailureException: Error occurred in the phase: 'Writing' with sqlState: '23000', errorCode: '1', errorMessage: ORA-00001: unique constraint (C##BUILDUSER.PK_AGE) violated

https://docs.oracle.com/error-help/db/ora-00001/
  at io.cdap.cdap.api.exception.ProgramFailureException$Builder.build(ProgramFailureException.java:229)
  at io.cdap.cdap.api.exception.ErrorUtils.getProgramFailureException(ErrorUtils.java:198)
  at io.cdap.plugin.common.db.DBErrorDetailsProvider.getProgramFailureException(DBErrorDetailsProvider.java:196)
  at io.cdap.plugin.common.db.DBErrorDetailsProvider.getExceptionDetails(DBErrorDetailsProvider.java:170)
  at io.cdap.cdap.etl.common.ErrorDetails.handleException(ErrorDetails.java:75)
  ... 14 common frames omitted
Caused by: java.sql.BatchUpdateException: ORA-00001: unique constraint (C##BUILDUSER.PK_AGE) violated

https://docs.oracle.com/error-help/db/ora-00001/
  at oracle.jdbc.driver.OraclePreparedStatement.generateBatchUpdateException(OraclePreparedStatement.java:10528)
  at oracle.jdbc.driver.OraclePreparedStatement.executeBatchWithoutQueue(OraclePreparedStatement.java:10171)
  at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10044)
  at oracle.jdbc.driver.OracleStatement.executeBatch(OracleStatement.java:5208)
  at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:288)
  at io.cdap.plugin.db.sink.ETLDBOutputFormat$1.close(ETLDBOutputFormat.java:99)
  at io.cdap.cdap.etl.spark.io.TrackingRecordWriter.close(TrackingRecordWriter.java:46)
  at io.cdap.cdap.etl.spark.io.StageTrackingRecordWriter.close(StageTrackingRecordWriter.java:65)
  ... 13 common frames omitted
2025-01-29 04:09:03,114 - WARN  [Executor task launch worker for task 0.0 in stage 0.0 (TID 0):i.c.p.d.s.ETLDBOutputFormat@106] - java.sql.SQLRecoverableException: ORA-17008: Closed connection
https://docs.oracle.com/error-help/db/ora-17008/
  at oracle.jdbc.driver.PhysicalConnection.requireOpenConnection(PhysicalConnection.java:12507)
  at oracle.jdbc.driver.PhysicalConnection.getAutoCommit(PhysicalConnection.java:2640)
  at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:2768)
  at io.cdap.plugin.db.sink.ETLDBOutputFormat$1.close(ETLDBOutputFormat.java:104)
  at io.cdap.cdap.etl.spark.io.TrackingRecordWriter.close(TrackingRecordWriter.java:46)
  at io.cdap.cdap.etl.spark.io.StageTrackingRecordWriter.close(StageTrackingRecordWriter.java:65)
  at org.apache.spark.internal.io.HadoopMapReduceWriteConfigUtil.closeWriter(SparkHadoopWriter.scala:373)
  at org.apache.spark.internal.io.SparkHadoopWriter$.$anonfun$executeTask$2(SparkHadoopWriter.scala:150)
  at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1550)
  at org.apache.spark.internal.io.SparkHadoopWriter$.executeTask(SparkHadoopWriter.scala:135)
  at org.apache.spark.internal.io.SparkHadoopWriter$.$anonfun$write$1(SparkHadoopWriter.scala:88)
  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
  at org.apache.spark.scheduler.Task.run(Task.scala:136)
  at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548)
  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1505)
  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.base/java.lang.Thread.run(Thread.java:829)


POST v3/namespaces/{namespace-id}/apps/{app-id}/workflows/DataPipelineWorkflow/runs/{run-id}/classify
[
  {
      "stageName": "Oracle2",
      "errorCategory": "Plugin-\u0027DB Integrity Constraint Violation\u0027-\u0027Oracle2\u0027",
      "errorReason": "ORA-00001: unique constraint (C##BUILDUSER.PK_AGE) violated\n\nhttps://docs.oracle.com/error-help/db/ora-00001/. For more details, see https://docs.oracle.com/en/error-help/db/ora-index.html",
      "errorMessage": "Error occurred in the phase: \u0027Writing\u0027 with sqlState: \u002723000\u0027, errorCode: \u00271\u0027, errorMessage: ORA-00001: unique constraint (C##BUILDUSER.PK_AGE) violated\n\nhttps://docs.oracle.com/error-help/db/ora-00001/",
      "errorType": "USER",
      "dependency": "true",
      "errorCodeType": "SQLSTATE",
      "errorCode": "23000",
      "supportedDocumentationUrl": "https://docs.oracle.com/en/error-help/db/ora-index.html"
  }
]
image

@psainics psainics added the build label Dec 23, 2024
@psainics psainics self-assigned this Dec 23, 2024
@psainics psainics marked this pull request as ready for review January 24, 2025 04:41

@Override
protected ErrorType getErrorTypeFromErrorCode(int errorCode, String sqlState) {
int prefixLength = 6;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add comments what is this logic exactly doing?

@psainics psainics merged commit 729c26a into data-integrations:develop Jan 29, 2025
8 of 10 checks passed
@psainics psainics deleted the fem/oracledb branch January 29, 2025 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants